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ABSTRACT 



In choosing statistics texts for social science 



students, both the matter and the mode of presentation must be 
considered. Selecting a statistics text is not as straightforward as 
selecting a text in the student's major field. Five instruments were 
developed to explore aspects of selection of statistics texts. These 
are: (1) a student survey for currently used textbooks; (2) an 
instructor survey for currently used textbooks; (3) an instructor 
survey of what the ideal text would be; (4) an expert evaluation 
instrument that may be used on any statistics textbook; and (5) an 
instrument covering relevant objective information about any 
statistics textbook. The decision to use a student evaluation was 
based on the importance of judging a book's utility at least partly 
on its ability to reach its intended audience. All five instruments 
are appended. (Contains 12 references.) (SLD) 
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How to choose textbooks for college students presents 
different problems for instructors than choosing textbooks for the 
lower grades. At the college level, students are assumed to have 
enough knowledge, ability and motivation to overcome deficiencies 
in style in a text if the content of the book meets the needs of 
a course. At the same time, students at the college level have a 
better understanding of what constitutes a well-written and. 
comprehensive textbook, and may therefor judge textbooks more 
stringently. 

The problem of choosing a textbook is compounded when the 
subject matter is not in the main area of concentration for the 
intended college audience. Presentation then becomes much more 
important in order to compensate for a lack of background in the 
student. Thus, in choosing statistics textbooks for social science 
students, both the matter and the mode of presentation have to be 
considered. Choosing an effective statistics text can also be a 
difficult task for instructors because the instructors are unlikely 
to feel "mach anxiety," whereas their students may not have had a 
mathematics course in many years, or otherwise may not feel 
comfortable with mathematical thinking. 

The aspects of how instructors should evaluate and compare 
college texts in general has been discussed in several articles, 
although this topic has not received the wide treatment that 
selection of texts for the lower grades has. Several authors have 
offered lists or forms to direct overall evaluation methods for 
textbooks (Prosser & Bondavalli, 1978; Talmage, 1981; Redei, 1984). 



Others have focused on single aspects of a textbook (e.g. pictures 
in Smith & Elifson, 1986; readability in Hollabaugh, 1989). Stang 
(1985) focused on whether college students could consistently rate 
textbooks, and, in the process, discusses depth, breadth and 
balance as important aspects in evaluating textbooks. 

Several authors have presented topics relating to statistics 
courses. Some have concentrated on a single subject as it is 
covered in textbooks (e.g. evaluation methods in Leviton & Cook, 
1983; research methods in Schutt, 1987). Chervany et al (1977) 
discusses how to set up meaningful evaluations of the effectiveness 
of a course. Cobb (1987) compares 16 statistics textbooks on 
several different measures, and in the process emphasizes judging 
the textbook as a tool and not simply as a product. Aiken (1990) 
has discussed what should be included in a statistics course. 
However, the particular case of how to systematically select 
statistics books for social science students has not been addressed 
to date. 

In the course of research conducted with Michael Harwell of 
the University of Pittsburgh, we have developed a series of five 
instruments intended to explore several aspects of such a 
selection. It should be noted that many of the categories of 
questions in these instruments were inspired by the previously 
named articles, and our debt to their groundwork is extensive. 
However, choosing a statistics text for social science students is 
not as straightforward a task as choosing a text for students in 
their major field of study, and therefore warrants a specialized 



series of instruments ♦ 

The instruments we have developed range from a general survey 
for instructors and students who are currently using a statistics 
textbook in a course, to particular instruments that may be used 
on textbooks not currently in use. The instruments are designed 
so that data obtained from their administration could be useful to 
broad-range researchers, to departments trying to choose a 
textbook, or to writers and publishers of new statistics texts . 

The five instruments are reproduced in whole in the appendix. 
These are: 1) a student survey for currently-used textbooks, 2) an 
instructor survey for currently-used textbooks, 3) an instructor 
survey of what an ideal statistics textbook would be like, 4) an 
expert evaluation instrument that may be used on any statistics 
textbook, and 5) an instrument covering relevant objective 
information about any statistics textbook. 

1. THE STUDENT SURVEY 

This survey is aimed at social science students currently 
enrolled in a statistics course. The way students respond to 
statistics textbooks is of interest for several reasons. How 
effectively the textbook is put to use by students will be affected 
by their attitude towards it. How closely student and instructor 
responses match in evaluating the textbooks will reflect both on 
how validly students can assess textbooks and how well instructors 
understand their students. 

We have included questions on demographic information, such 



as age and student status, that may be relevant to, or may be of 
interest in ruling out as influencing, students 1 ratings of the 
texts. Of particular note are the questions on date, grade and 
level of the students 1 last mathematics courses. A common 
criticism of statistics courses for social science students is that 
they presume a higher level of mathematical ability than is 
commonly the case among these students. The questions about 
mathematics will help determine the average mathematical 
background, and whether this background influences the rating of 
a textbook. 

The questions asking students to characterize the 
explanations, writing style and overall emphasis in the textbook 
are both to check students 1 agreement on these matters and to 
channel the students 1 thinking into systematic modes. This adds 
to the systemic validity of the instrument. 

Several changes worth noting were made due to responses from 
the piloting of this instrument. A number of graduate students 
wrote in that they did not use the assigned textbook at all after 
the first couple of weeks. Some of these students further 
specified that they turned to undergraduate texts or computer 
software manuals as being more understandable. We thereupon added 
questions and alternatives choices designed to capture this lack 
of use to the survey. 

The student questionnaire initially contained questions on how 
important the presentations of certain key concepts were in the 
textbook. However, in piloting, a large proportion of students 
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responded that they did not know how important the concepts were; 
one student who did respond with ratings wrote that he or she was 
not sure if the responses reflected the book's or the instructor's 
emphasis. This section of the questionnaire was also the one most 
commented upon by other reviewers. Taking all of these occurrences 
into consideration, we decided to drop the section on concept 
presentation entirely from the student questionnaire, retaining it 
only in the instructors' version. 

2. THE INSTRUCTOR SURVEY 

This survey is aimed at instructors currently teaching 
statistics to social science students. It is designed to be used 
in conjunction with the student survey, and the questions therefore 
are designed to parallel that survey as much as possible so as to 
facilitate data comparison. Most of the comments about the student 
survey are relevant here. 

This survey contains a number of questions about the 
incorporation of statistical software usage into the course. The 
great improvements in the ability of computers to manipulate data 
sets, along with the development of affordable statistical software 
programs, have made the use and choice of such programs a 
particularly relevant concern in designing statistics courses. 

There are also several open-ended questions pertaining to 
instructors' opinions on the strength and weaknesses of the text 
in question. We took the open-ended, as opposed to the 
categorical, approach so as to be able to capture any elements that 

5 



may not have been covered elsewhere in the survey, and because the 
expected sample size would be manageable enough to allow the 
responses to be categorized post hoc without a major commitment of 
time* 

3. THE IDEAL STATISTICS TEXTBOOK SURVEY 

This survey is intended to assess what instructors would most 
like to see in a statistics textbook intended for social science 
students. This survey, while of general interest to instructors 
of statistics courses, could also be used to help design a new 
textbook, or to help faculty focus on what is important to them in 
a textbook before evaluating what is actually available. 

This survey consists largely of open-ended questions. We 
considered and rejected the strategy of devising categories for the 
majority of these questions so as to encourage as much variation 
and precision of opinion as possible. The analysis of these 
questions should' not be overly time-consuming, given the probable 
small sample size of any administration. 

In several cases where we felt giving choices would be more 
helpful than limiting to subjects, such as in characterizing the 
nature of ideal homework problems, we have provided specific 
categories in addition to an open-ended option. 

4. THE EXPERT EVALUATION INSTRUMENT 

This instrument is intended to be used on statistics textbooks 
not currently being used to teach a course. The instrument 



presupposes that the people using the instrument have a good basic 
knowledge of social science statistics, and, if possible, have* 
already taught such a course. 

The challenge in designing this instrument was to define a 
task that would not be prohibitive in length while still allowing 
representative access to the textbook in question. We therefore 
chose four topics that are typically covered in no more than a 
section of a chapter in statistics textbooks, and that should be 
included in every textbook under review. These topics are divided 
between conceptual areas — type 1 and type 2 error, ?.nd the 
conceptual . overview of statistical inference — and statistical 
application areas — the chi-square test of independence, and 
interaction in two-way analysis of variance. Each topic is 
measured through parallel ratings and open-ended comments. 

The highest rating possible in this instrument on overall 
presentation of a topic in the textbook is, "just how I would do 
it." This is more than a humorous touch: it is meant to emphasize 
the subjective nature of any rating of presentation method. 
Despite the subjectivity of such a rating, how closely a 
presentation approaches the way an instructor would teach the 
subject is still an important element in evaluating the utility of 
a textbook. 

5. THE OBJECTIVE INFORMATION INSTRUMENT 

This instrument is intended to be used in conjunction with the 
expert evaluation instrument. It covers the aspects of a textbook 
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that are relevant to making a choice but that are not related to 
content, such as the presence of a glossary in the textbook or the 
contents of the appendices. This instrument would need to be 
completed only once for any given textbook. 

Included in this instrument is a readability measure, based 
on a combination of the average length of sentences and the 
proportion of large words in the text. This particular measure, 
the FOG scale, was chosen over several others for its ease of use 
and its yielding of scores, stated as year of education the text 
is suitable for, in the expected undergraduate and graduate range. 

DISCUSSION 

f The degree of validity of student evaluations on textbook 
evaluation in areas where they are not expert may be questionable. 
The decision to include a survey for students as a method to 
evaluate textbooks in this set of instruments was therefore not 
based on students 1 knowledge of the subject, but rather on the 
importance of judging a book f s utility at least partly on its 
ability to reach its intended audience (see Huberty, White and 
Wolins, 1985). 

The fact that college students 1 overall recommendations for 
a text may be based on the style, rather than the content, of a 
text should not be trivialized. The better written and more 
understandable a text is, the more likely students are to use it 
during a class, and to refer to it after taking the class. The 
more likely it is that students use a text, assuming basic content 



is adequate, the better served the subject matter will be. 

An exceptionally comprehensive text that is difficult to read 
may leave an impression even with conscientious students that they 
have no control over how to learn the content, and that their only 
hope, of understanding it is to have the instructor, as a 
professional, teach them point by point in class. On the other 
hand, an easier to read, although less comprehensive, text would 
allow students to explore points made in class in more detail, and 
leave the impression on students that they have the ability to 
explore the subject themselves. 

It may be difficult for an instructor to adequately assess the 
style of a textbook, since intimate knowledge of the material may 
make points seem clear that would be less clear to someone with a 
sparser background. This makes it especially useful to have some 
sort of student input whenever textbook quality or choice is being 
assessed. 

CONCLUSION 

These five instruments were designed to fulfill a dual 
purpose: taken in sets, they could assist general research directed 
towards comparing textbooks; and taken together, they could assist 
a department trying to choose a textbook for use in a course. 

In terms of general research, the student and instructor 
surveys on textbooks currently in use can be used together to draw 
comparisons both between books and between student and instructor 
perceptions. The expert evaluation and the objective information 
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instruments can be used together to define discrete areas with 
which to compare textbooks not currently being used by the 
evaluators. The ideal textbook survey could be used to set a 
benchmark against which to compare the results of the other 
instruments. 

In terms of assisting a department's selection of a textbook, 
the five instruments may be used together. The student and 
instructor surveys may be used to evaluate the textbook currently 
in use; the expert evaluation and the objective information 
instruments may be used to evaluate textbooks under consideration; 
and the ideal textbook survey may be used to focus the faculty's 
ideas of what should be expected of a textbook. 

The choice of a textbook can have a large effect on what is 

learned in a course. The more accessible the textbook's style, the 

more it will be used; the more complete the textbook's content, the 

more valuable it is when the class is over. The importance of the 

textbook is magnified in statistics classes aimed at social science 

students, because of the "math anxiety" so prevalent among these 

students. Huberty, White and Wolins state as the third issue on 

their list of issues for future researchers to address as: 

3 . The challenge implicit in the need to motivate people 
in required service courses who are not quantitatively 
oriented . Are these courses really the "necessary evil" 
they are sometimes claimed to be, or are some of those 
students the future public school teachers and university 
administrators who will have a certain impact on the 
future of our profession? (p. 171) 

We hope that these instruments will have a part in changing that 

"necessary evil" into sincere appreciation, and even into delight. 
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AUTHOR OF TEXT 
COURSE NUMBER : 
INSTRUCTOR : 



SECTION NUMBER : 



THIS IS AN EVALUATION OF THE WAY TEXTBOOKS ARE USED IN STATISTICS 
COURSES THAT ARE AIMED AT SOCIAL SCIENCE STUDENTS. YOUR RESPONSES 
WILL BE KEPT CONFIDENTIAL. 



PLEASE CIRCLE YOUR ANSWER TO THE QUESTIONS BELOW, 
la. What is you student status? 

a. Graduate 

b. Undergraduate 



lb. What is your major/field? 

a. Psychology 

b. Education 

c. Sociology 

d. Other: Specify 

2. If you are a graduate student, what was your undergraduate 
major? 

a. Humanities 

b. Behavioral science, social science or education 

c. Physical or biological science 

d. Mathematics or computer science 

e. Other: Specify 



3. What is your gender? 

a. male 

b. female 

4. What is your age? 

a. 20-24 

b. 25-29 

c. 30-34 

d. 35-39 

e. 40 or older 



5. What is the highest level of mathematics that you have studied? 

a. General mathematics 

b. Algebra/Trigonometry . 

c. Calculus 

d. Beyond calculus 



When did you take your last mathematics course? 

a. In the last two years 

b. Two to five years ago 

c. More than five years ago 

What was your grade in your last mathematics course? 

a . A or B 

b. Below B 

c. Don't remember - . 

14 



8. Have you ever taken a statistics course before? 

a . Yes 

b. No 



9. How does your instructor expect you to use the textbook? 
Check all that apply. • 

a . As a source to learn from 

b . As a resource to supplement the class lessons 

c. As the major source for homework problems 

d . Other- -Specif y : 

10. How do you actually use the textbook? Check all that apply. 
a. As a source to learn from 

b. As a resource to supplement the class lessons 

c. As the major source for homework problems 

d . Other- - Specif y : 

e . Did not use the book 



11. How would you describe the mathematics level assumed in the 
textbook: (Circle the number) 

1 2 3 4 5 

very about very 

easy right difficult 

12. How would you describe the reading level of the textbook: 
(Circle the number) 



1 2 3 4 5 

very about very 

easy right difficult 

13 . How closely does the table of contents of the textbook match 
the order of presentation of the materials in class? 

a . The sequence is identical . 

b. There are one or two topics out of sequence or missing 

c . There are three or more topics out of sequence or 

missing 

14 . How independent are the chapters from each other? 

a. Most chapters rely heavily on previous chapters 

b. Most chapters rely somewhat on previous chapters 

c. Most chapters may stand on their own 

15. How would you describe the homework problems contained in the 
text? Check all that apply. 

a. Based on actual cited studies 

b. Hypothetical but realistic in set-up 

c. Challenging to do 

d. Interesting and substantive 

e. Too difficult to do 

f . Too easy 

g. Have not had to do any problems from the book 

h. Help strengthen skills and knowledge covered by book 



16. The text is easy to understand and follow. (Circle the number) 
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1 2 

strongly 

disagree 



neutral 



strongly 
agree 
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Explanations rely heavily on 
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Check all that apply 

a. examples 

b. proofs 

c. verbal descriptions 

d . diagrams or sketches 

e . graphs 

f . analogies 

Check all that apply: The writing style is 

a. terse 

b . wordy 

' c . simple 

d. balanced 

e. distracting 

f. such that it does not draw attention away 

from the subject 

g. comprehensive (complete) 

h. easily understandable 

i. flows easily 

j . direct 

k. uneven (terse some places, wordy in others) . 

19. Check all that apply: Overall emphasis is on 

a . computations 

b . actual cited studies 

c. realistic examples 

d . theory 

e. applications 

f . comprehension 

g. developing research skills 

h. computer usage 

i . experimental design issues 

j . practical problem- solving 

FOR THE FOLLOWING ITEMS, CIRCLE THE NUMBER THAT BEST REPRESENTS 
YOUR OPINION. 

.20. Coverage of topics is very complete, (depth) 



1 2 
strongly 
disagree 



neutral 



strongly 
agree 



9 

do not 
know 



21. Coverage of topics is proportional to their importance 



1 2 

strongly 

disagree 



neutral 



16 



strongly 
agree 



9 

do not 
know 



22. The variety of topics covered is broad 

1 2 3 4 5 9 

strongly neutral strongly do not 

disagree agree know 

23 . The text clearly shows how to systematically perform the 
computations necessary for using the statistical methods that you 
are studying. 

1 2 3 4 5 

not at all incomplete minimal thorough extensive 

(but complete) 

24. The text clearly shows how the computations are linked to the 
theory of the statistical method. 

1 2 3 4 5 

not at all incomplete minimal thorough extensive 

(but complete) 

25. How useful was the workbook in supplementing the materials 
contained in the text? 



1 2 3 4 5 9 

not somewhat very no 

useful useful useful workbook 



26. How frequently did you do the exercises in the text? 

a. frequently 

b. occasionally 

c. seldom/never 

26. Would you recommend this book to students who are trying to 
learn statistics on their own? 

a. Strongly recommend 

b. Recommend 

c . Not recommend 

27. Would you recommend the use of this book for a statistics 
course? 

a . Yes 

b. No 

28. Did you make use of a statistical software manual? 

a . Yes 

b. No 

29. If yes to question 28- For learning statistics the computer 
manual was : 

a. More useful than the text 

b. Useful in addition to the text 

c. Less useful than the text 

d. Not useful 

ERIC 1 



INSTRUCTOR NAME: 
AUTHOR OF TEXT_ 
COURSE NUMBER: 



UNIVERSITY: 

SECTION NUMBER: 



THIS IS AN EVAHJATION OF THE WAY TEXTBOOKS ARE USED IN STATISTICS COURSES THAT 
ARE AIMED AT SOCIAL SCIENCE STUDENTS. YOUR RESPONSES WILL BE KEPT CONFIDENTIAL. 
PLEASE USE ONE FOPM FOR EACH TEXTBOOK YOU EVALUATE. 

PLEASE CIRCLE YOUR ANSWER TO THE QUESTIONS BELOW. 

la. What is the student status of the course this book is used in? 

a. Graduate 

b. Undergraduate 

lb. In which field is your department? 

a. Psychology 

b. Education 

c. Sociology 

d. Other: Specify 

2. What is your faculty position? 

a. Professor 

b. Instructor/Adjunct 

c. Graduate Student with teaching duties 

d. Other: 



3. What is your gender? 

a. Male 

b. Female 

4. Hew long have you been teaching? 

a. 1-5 years 

b. '- 6-10 years 

c. 11-15 years 

d. 16-20 years 

e. more than 20 years 

5. What would you say is the average level of mathematics that your students 
have completed? 

a. General mathematics 

b. Mgebra/Trigorxanetry 

c. Calculus 

d. Beyond calculus 

6. Hew long ago did your average student take a mathematics course? 

a. In the last two years 

b. Two to five years ago 

c. More than five years ago 

7 . How would you characterize the average attitude towards mathematics of your 
students? 

a. Good 

b. Indifferent 

c. Poor 



is 



8. Have you ever taught a statistics course from a book other than the one you 
are presently evaluating? 

a. Yes 

b. No 

9. How do you expect your stixients to vise the textbook? Check all that apply. 

a. As a source to learn frcm 

b. As a resource to supplement the class lessons 

c. As the major source for homework problems 

d. Other — Specify: 

10. Hew do you think they actually use the textbook? Check all that apply. 

a. As a source to learn from 

b. As a resource to supplement the class lessons 

c. As the major source for homework problems 

d. Other — Specify: 

e. Did not use the book 



11. Hew would you describe the mathematics level assumed in the textbook: 
(Circle the number) 

1 2 3 4 5 

very about very 

easy ri^ht difficult 

12. Hew would you describe the reading level of the textbook: (Circle the 
number) 

1 2 3 4 5 

very about very 

easy right difficult 



13. Hew closely does the table of contents of the textbook match the order of 
presentation of the materials in class? 

a. The sequence is identical. 

b. There are one or two topics out of sequence or missing 

c. There are three or more topics out of sequence or missing 

14. How independent are the chapters frcm each other? 

a. Most chapters rely heavily on previous chapters 

b. Most chapters rely somewhat on previous chapters 

c. Most chapters may stand on their cwn 



15. How would you describe the homework problems contained in the text? Check 
all that apply. 

a. Based on actual cited studies 

b. Hypothetical but realistic in set-up 

c. Challenging to do 

d. Interesting and substantive 

e. Too difficult to do 

f . Too easy 

g. Have not looked at the problems fran the book 

h. Help strengthen skills and knowledge covered by book 
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16. The text is easy to understand and follow. (Circle the number) 

1 2 3 4 5 

strongly neutral strongly 

disagree agree 

17. Check all that apply: Explanations in the book rely heavily on 

a. examples 

b. proofs 

c. verbal descriptions 

d. diagrams or sketches 

e. graphs 

f. analogies 

18. Check all that apply: The writing style in the book is 

a. terse 

b. wordy 

c. siitple 

d. balanced 

e. distracting 

f . such that it does not draw attention away from the subject 

g. canprehensive (complete) 

h. easily understandable 

i. flews easily 

j . direct 

k. uneven (terse scire places, wordy in others) . 

19. Check all that apply: Overall emphasis in the book is on 

a. computations 

b. actual cited studies 

c. realistic examples 

d* theory 

e. applications 

f. ocmprehension 

g. developing research skills 

h. cenputer usage 

i. experimental design issues 

j. practical problem -solving 

FOR THE FOIiCWING ITEMS, CIRCLE IHE NUMBER THAT BEST REPRESENTS YOUR OPINION OF 
THE CONTENT OF THE BOOK. 

20. Coverage of topics is very complete, (depth) 

1 2 3 4 5 9 

strongly neutral strongly do not 

disagree agree know 

21. Coverage of topics is proportional to their itrportance 

1 2 3 4 5 9 

strongly neutral strongly do not 

disagree agree know 
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22. The variety of topics covered is broad 

1 2 3 4 5 9 

strongly neutral strongly do not 

disagree agree knew 

23. The text clearly shows hew to systematically perform the computations 
necessary for using the statistical methods that you are studying. 

1 2 3 4 5 

not at all incomplete minimal thorough extensive 

but complete 

24 . The text clearly shows hew the computations are linked to the theory of the 
statistical method. 

1 2 3 4 5 

not at all , incotplete minimal thorough extensive 

but complete 

25. Hew useful was the workbook in supplementing the materials contained in the 
text? 

1 2 3 4 5 9 

not somewhat very no 

useful useful . useful workbook 

26. Hew frequently did you assign exercises in the text? 

a. frequently 

b. occasionally 

c. seldcw/never 

27 . Would you recommend this book to students who are trying to learn statistics 
on their own? 

a. Strongly recommend 

b. Recommend 

c. Not recommend 

28. Would you recommend continuing the use of this book for a statistics course? 

a. Yes 

b. No 

29. Did you make use of a statistical software manual? 

a. Yes 

b. No 

30. Did you use supplementary materials other than a software manual? 

a. No 

b. Yes, for enrichment 

c. Yes, because the text was inadequate on one or more topics 
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31. What would you characterize as the main strength of the book? Was this the 
reason the book was selected for this class? 



32 . What would you characterize as the main weakness of the book? Is this also 
what your students would say? 



33. Does the text have materials relating to computer implementation 
of statistical procedures (e.g. sample SAS programs)? 

a) yes 

b) no 

IF THE ANSWER TO QUESTION 33 IS YES. . . 

34. Hew useful do you find these materials? 

a) I have integrated them into class assignments. 

b) I make reference tt these materials but do not 
use them in assignments. 

c) I do not use these materials. 

35. If you have responded c) to the above question please indicate 
why you do not make use of these materials. Circle all that 
apply. 

a) lack of familiarity with the stat package used 

b) Inadequacy of materials-please elaborate below 

c) Other-please elaborate below 

Comments for b) or c) : 



INSTMJCTCR: 



UNIVERSITY: 



2. 



> SURVEY IS TO ASSESS THE CHARACTERISTICS OF AN "IDEAL" STATISTICS TEXTBOOK 
SOCIAL SCIENCE STUDENTS. 

USE RESPOND TO ALL QUESTIONS IN TEEMS OF YOUR PERCEPTIONS OF AN 
SAL" TEXTBOOK. 

Hew would you describe the mathematics level assumed in the ideal textbook? 
Please be as specific as possible in terms of either course level or 
particular topics and skills level. 



How would you describe the reading level of the textbook? Again, please 
be specific in terms of grade level or types of journals and other 
reading materials that you would consider of comparable reading level. 



What should be the structure or order of materials presented? (This 
question can be answered by taking a copy of an existing table of 
contents and editing it to reflect your ideal table of contents, or 
by losing the letters of the topics on the right.) 



What proportion of the text per topic would reflect their relative iirportance 
in the curriculum? Indicate proportions next to the topics listed in 
response to question three. 



A. 
B. 
C. 
D. 
E. 
F. 
G. 
H. 
I. 
J. 
K. 
L. 
M. 
N. 
0. 



Analysis of Variance 
Factor analysis 
Multivariate analysis 
Research methods 
Evaluation research 
Experimental design 
Time series 
Analysis of covariance 
Pcwer analysis 
Ncrparametric proaackires 
Chi-square analyses 
Z-tests 
T-tests 
Correlation 
Regression 
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5. Hew ir^^erdent/ interrelated should the chapters be from each other? 
Particular illustrations for each conment would be helpful. 



6. Hew would you describe the homework problems contained in the text? Check 
all that apply, STAR THE SINGLE MOST IMPORTANT • 

a. Based on actual cited studies 

b. Hypothetical but realistic in set-up 

c Challenging to do 

•_ d. Interesting and substantive 

e. No need for problems 

f . Help strengthen skills and knowledge covered by book 

g. Other: 



7. Check all that apply. Star the single most inportant: Explanations in the 
book should rely heavily on 

a. examples 

b. proofs 

c. verbal descriptions 

d. diagrams or sketches 

e. graphs 

f. analogies 

g. other: 



8. Check all that apply. Star the single most iirportant: Overall enphasis in 
the book should be on 



a. 


computations 


b. 


actual cited studies 


c. 


realistic examples 


d. 


theory 


e. 


applications 


f. 


comprehension 


g. 


developing research skills 


h. 


conputer usage 


i. 


experimental design issues 


j. 


practical problem-solving 


k. 


other: 



9. What aspects of the text are most crucial to assure adequate breadth? 
(e.g. breadth of illustrations, statistical procedures, assortment of 
designs) 
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10. What role should hand (not by cxxtputer) cctrputational competence play in the 
overall focus of the text? Please list specific benchmarks such as being 
able to reproduce outcomes on oarputerized output, being able to select and 
apply statistical formulas in an unreal ist ical ly simplified example, explain 
how to use formulas, interpreting computer's computations etc. 



11. What role, if any, should a supplementary workbook play? What should 
it provide the students that the text does not already provide? 



12. What role, if any, should computer language instruction, worked computer 
problems and illustrations play in the text? 



13. We all have colleagues who have different textbook perspectives. Which 
of your characterizations of the ideal text do you think would be the most 
controversial to your colleagues? 



EVAIUftTOR 



AI7IHOR 



THIS INSTRUMENT IS INTENDED AS A MEANS OF COMPARISON BETWEEN STATISTICS TEXTBOOKS 
USED FOR SOCIAL SCIENCE STUDENTS. PLEASE FILL OUT A SINGLE FOPM FOR EACH 
TEXTBOOK EVALUATED. 

BEFORE COMPLETING THIS FOPM, EVAHJATORS ARE EXPECTED TO HAVE READ THE SECTIONS 
IN THE TEXTBOOK ON: 1) INTERACTION IN TWO-WAY ANALYSIS OF VARIANCE; 2) CONCEPTOAL 
OVERVIEW OF STATISTICAL INFERENCE; 3) TYPE I AND TYPE II ERRORS; AND 4) CHI-SQUARE 
TEST OF INDEPENDENCE. 

For the interaction in two-way analysis of variance section: 

1. Balance: is the time spent on topics in this section proportional to the 
iirportanoe of the topics? 

1 2 3 4 5 

^ completely adequate if excellent 

inappropriate supplemented in class use of emphasis 

COMMENTS: 



2. Breadth: are all the important topics, and the important aspects of those 
topics, covered in this section? 

1 2 3 4 5 

major parts adequate if very 

left out supplemented in class ccarplete 

CCMMENIS: 



3. Depth: are the topics covered in sufficient detail in this section, or is 
the detail too excessive or lacking? 

1 2 3 4 5 

depth completely adequate if perfect 

inappropriate compensated for airount 

(too much or little) in class of detail 

COMMENTS: 



4. Hew is the topic presented in this section? For example, is an example 
given first or is a general question posed? Is the theory presented before 
or after any equations? Is there a worked example? Are actual studies 
cited, and, if so, where in the presentation? 



5. Overall presentation rating: 

1 2 3 4 5 

very awkward adequate just how I 

would do it 
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For the conceptual overview of statistical inference section; 

1. Balance: is the time spent on topics in this section proportional to the 
iitportance of the topics? 

1 2 3 4 5 

completely adequate if excellent 

inappropriate supplemented in class use of emphasis 

OCWMENTS: 



2. Breadth: are all the important topics, and the important aspects of those 
topics, covered in this section? 

1 2 3 4 5 

major parts adequate if very 

left out supplemented in class complete 

CEMENTS: 



Depth: are the topics covered in sufficient detail in this section, or is 
the detail too excessive or lacking? 



1 2 
depth completely 

inappropriate 
(too much or little) 



adequate if 
compensated for 
in class 



5 

perfect 
amount 
of detail 



COMMENTS: 



4. Hew is the topic presented in this section? For example, is an example 
given first or is a general question posed? Is the theory presented before 
or after any equations? Is there a worked example? Are actual studies 
cited, and, if so, where in the presentation? 



5. Overall presentation rating: 



1 2 3 4 5 

very awkward adequate just how I 

would do it 



For the type I and type II errors section; 

1. Balance: is the time spent on topics in this section proportional to the 
iirportanoe of the topics? 

1 2 3 4 5 

completely adequate if excellent 

inappropriate supplemented in class use of emphasis 

OCMMENTS: 



2. Breadth: are all the important topics, and the important aspects of those 
topics, covered in this section? 

1 2 3 4 5 

major parts adequate if very 

left out supplemented in class caiplete 

CCMMENIS: 



3. Depth: are the topics covered in sufficient detail in this section, or is 
the detail too excessive or lacking? 

1 2 3 4 5 

depth completely adequate if perfect 

inappropriate compensated for amount 

(too much or little) in class of detail 

(TMMENIS: 



4. Hew is the topic presented in this section? For example, is an example 
given first or is a general question posed? Is the theory presented before 
or after any equations? Is there a worked example? Are actual studies 
cited, and, if so, where in the presentation? 



5. Overall presentation rating: 

1 2 3 4 5 

very awkward adequate just how I 

would do it 
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For the chi-square test of independence section: 

1. Balance: is the time spent on topics in this section proportional to the 
importance of the topics? 

1 2 3 4 5 

completely adequate if excellent 

inappropriate supplemented in class use of emphasis 

OCMMENIS: 



2. Breadth: are all the important topics, and the important aspects of those 
topics, covered in this section? 

1 2 3 4 5 

major parts adequate if very 

left out supplemented in class complete 

OCMMEOTS: 



Depth: are the topics covered in sufficient detail in this section, or is 
the detail too excessive or lacking? 



1 2 
depth completely 

inappropriate 
(too much or little) 



adequate if 
compensated for 
in class 



perfect 
amount 
of detail 



COMMENTS: 



4. Hew is the topic presented in this section? For example, is an example 
given first or is a general question posed? Is the theory presented before 
or after any equations? Is there a worked example? Are actual studies 
cited, and, if so, where in the presentation? 



5. Overall presentation rating: 



1 2 3 4 5 

very awkward adequate just how I 

would do it 
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THIS INSTRLMEOT IS INTENDED TO BE AN OBJECTIVE MEASURE OF CERTAIN IMPORTANT 
ASPECTS OF TEXTBOOK LAYOUT. PLEASE COMPLETE A SINGLE FDFM FOR EACH TEXTBOOK TO 
BE EVALUATED. 



1. Does the book have: a teacher's manual YES NO 

a student's workbook YES NO 

2. Check the internal structures present in the book: 

a. Detailed table of contents 

b. Introductions at the beginning of chapters 

c. Summaries at the end of chapters 

d. New terms listed at the end of chapters 

e. Glossary 

f . Labeled diagrams 

g. Answer key 

3. What does the Preface say about the: 

Intended audience: 
Purpose: 
Point of view: 
Suggestions for use: 



4. What is the average number of study questions per chapter? 

5. What, if any, corputer program is used in the book? 

6. List the topics of any appendices: 



7. Rea d ability: Compute the FOG index twice for each book and average the 
results. This index is intended to be a rough measure of the year of school 
the reading level is aimed at. Hcwever, note that it was not originally 
intended for use at college and graduate levels. See the next page for 
directions on hew to compute. 

AVERAGE OF TOE TWO FOG INDICES: 
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HOW TO COMPOTE THE FOG INDEX:* 

Count off 100 words, starting at the beginning of a sentence, . on each of the 
pages that divide the book into thirds, and continue until you each the end of 
a sentence (if the page is part of a problem set, or otherwise not typical, go 
to the first typical page) . 

1/3 thrcugh: Number of sentences: 

Number of words: 

Average number of words per sentence: 

Number of words with 3 or more syllables: 



FOG index: Sum of average number of words + number of 
words with 3 or more syllables 
Multiply by 0.4, round to nearest integer 



2/3 through: Number of sentences: 

Number of words: 

Average number of words per sentence: 
Number of words with 3 or more syllables: 

FOG index: Sum of average number of words + number of 
words with 3 or more syllables 
Multiply by 0.4, round to nearest integer 



* Adapted from Hollabaugh, M. (1989) . Textbook selection: Clearing the fog. 
Journal of College Science Teaching . 18, 5, 327-329. 
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